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A FREQUENCY CRITERION FOR THE EXISTENCE OF AN 
OPTIMAL CONTROL FOR ITO EQUATIONS* 



Nikolai Dokuchaev 



, Abstract 

The following optimization problem is considered. For the vector Ito equation 

dx{t) = [Ax{t) + hu{t)]dt + Cx{t)dw{t) 



with initial conditions x(0) = a it is required to find an optimal deterministic control vector 
u{t) e -^^^[(0, +00), R™] which minimizes the functional 

/"OO 

= / [Ex{tYGx{t)+u{tyVu{t)]dt. 







U 
O 



> 

^ ; A necessary and sufficient condition for the existence of a optimal control are formulated in 

(N ■ ... 

• the form of frequency inequalities for functions depending on the matrices A, b, C, G and V. 

It is shown that an optimal control vP{t) can be found by solving a certain linear-quadratic 
deterministic optimization problem. 

Key words: optimal control, frequency theorem, Ito equations 
^ , We consider the following optimization problem on a standard probability space (17, P): 

dx{t) = {Ax{t) + hu{t))dt + Cx{t)dw{t). (1) 

xq = a. (2) 

poo 

^u{■)] = / [Ex{t)^Gx{t) + u{tyTu{t)\dt = min (3) 

Here t > 0, dw{t) is a random walk adapted to a nondecreasing flow of cr-algebras J-{t) C 
xt is a random n-vector of states, u{t) is a non-random m-vector of controls and A, G = 
G^,r = and b are are constant matrices of respective order n x n, n x n, n x n, m x 
m and n x m. All the vectors and matrices in ([T]) to ([3]) are real and E denotes expectation. 



' Vestnik Lemgrad Umv. Mathematics. 16 (1984), pp. 41-47. Translated from Russian by H.H.McFaden. 
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The norm of a complex or real vector (matrix) z is understood to be the square root of the sum 
of of the squares of the moduli of its elements, and is denoted by \z\. Also, let = (E |^|^)^'''^. 
The random vector a is measurable with respect to the cr-algebra Tq^ is independent of dw{t) 
and satisfies E |ap < +00. Equation ([1]) is the Ito equation. 

It is assumed that ^ is a Hurwitz matrix and that for u{t) = the system in ([T|), ([2]) is 
exponentially stable in the mean square, i.e., there exist numbers c,e > such that |x(t)|2 < 
ce~^*|a|2 (Vt > 0); this holds, for example under the mildly restrictive conditions given in Levit 
and Yakubovich (1972). 

We establish a criterion for the existence of an optimal solution in the class U = 
L^[(0, +00), R™'] of deterministic measurable m-vector valued functions u{t) such that \u{t)\ S 
L2(0,+oo). 

If u^{t) is an optimal control, then 

$[n°(-)] ^ ^H-)] (Vn(-) G U). 

The proof of the proposed criterion is based on results obtained in Yakubovich (1975), and 
makes essential the use of the idea of a proof given in Yakubovich (1975) for ordinary differential 
equations. 

Consider the matrix- valued function g{X) = {iXI — A)^^. Here and below, i is the imaginary 
unit, A is in R^, and / is the identity matrix. Suppose that the matrix satisfies the equation 

1 f'°° 

e = G+— C^gi-X)^eg{X)Cd\. (4) 
^71" J -00 

We consider on x the Hermitian form 

F{x,u) = x*ex + u*Tu. (5) 

1. A preliminary result is the following theorem, which actually establishes a criterion for the 
existence of u^{t). 

THEOREM 1 If there exists an optimal control vP {t) , then 

F{g{X)bu,u) >0 (VA e R\Vu e C^) (6) 

If there exists a number 6 > such that 

F{g{X)bu,u) > 6\u\'^ (VA G R\ V^/ € C") (7) 

then there exists an optimal control u^{t), and it is unique to within equivalence. 
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PROOF. Let us consider the real Hilbert space Y = {y} = P) and the complex 

Hilbert space Yc = {y} = .F, P, C") of random n- vectors y P-equivalent vectors bemg 

identified) with respective inner products ^yjy2 and Eyly2. 

For an arbitrary Hilbert space H = {h} with inner product (/ii,/i2)) and for an interval 
T C R we denote by Z = L^{T, H) the Hilbert space of strongly measurable i?- valued functions 
h{t) such that G Lp'{T). If is a real (complex) space, then we regard Z as & real 

(complex) space. 

As usual, the inner product and the norm in Z are 



{hi[-)M{-))z = Y,ihi{t)M{t)) and \h{-)\z = {h{-),h{-))l 
Consider the Hilbert spaces 

X = L2[(-oo,+oo),yc], U = l2[(-oo,+oo),C™]- 
For all x{t) £ X, u{t) £U,a> 0, A G R, let 

x„(A) = [ e-'^^x{t)dt u„(A) = [ e-'^^u{t)dt. 

y2TT J -a v27r J -a 

For x{t) and u{t) we define the Fourier transforms Xa(A) and Ua(A) in a way similar to that in 

Yakubovich (1975) (Part I, §1) so that \x{-) — Xa{')\x — ^ and |n(-) — 'Sa(-)|{7 — ^ a a — > 

+00. 

We now consider the functions u{t) G U. Let x{t) satisfy ^ and ([2]) for t > 0. Then x{t) is 
a real random function. For m{t) = Ex(t) and M{t) = Ex(t)x(t)^ we have the relations 

m{t) = Am{t) + bu{t) 

M{t) = AM{t) + M{t)A^ + hu{t)m{ty + m{t)u{tyh^ + CM{t)C^. 

Let 

f + OO \ 



\z{t)\L, = ^ \z{trdt 

Then we can see that there are numbers cj > such that 

\M{t)\L, ^ ci\m{t)u{t)\Li + 4\a\l ^ C3\m{t)\L,.\u{t)\L2 + 4\a\l ^ c^kWlL + ^ikli (8) 
Equation ([1]) means that 

''^^dx{t) = [ e''^^[Ax{t)dt + bu{t)dt + Cx{t)dw{t)] 

a 







i\ / e-'^'x{t)dt + e~'^"x{a) - x(0). 



It follows from ([S]) that if u{t) G U, then |x(t)|2 S L^(0,+oo). We regard x{t) and u{t) as 
elements of X and U by setting x(t) = and = for t < 0. Let 



/„(A) = -^C I e-'^'x{t)dw{t), h^{\) = g{X)fa{X)- 

V ^TT Jo 



Then for a = 



iXxaia) = Axa{X) + buaiX) + /a(A) -=e '^°'x{a). 



Note that E/iq,(A) = 0; we have 

Ex„(A)*Gx„(A) = E{[g(A)te«(ArG[5(A)te,(A)] 

^ Re[5(A)6Ma(A)]*G5(A)e~*^"2;(a) 



27r 

+-^b(A)e-^^"x(a)]*G[5(A)e-^^"x(a)] + /i,(A)*G/i„(A)}. 



Hence for /3 > 



/ Bxc,{X)*Gxa{X)dX= [ Ua{X)*b^g{-XyGg{X)buc,{X)dX+ [ E /i„(A)*G/i(A)dA+^'(/3, a). 

(9) 

The function ^{(3, a) goes to zero uniformly in /? as a — )• +oo, because |x(a)|2 — > 0, the 
function |/a(A)|2 is bounded, and the functions |s'(A)| and \g{X)bua{X)\ are in oo,+oo). 
Let 

T^{G) = ^J C'^g{-X)'^Gg{X)CdX, 
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TiG) = ^ I C^g{-X)^GgiX)CdX. 
Then by a property of Ito integral, 

['^ Eho,iX)*GhaiX)dX= [ Bx{tyT^{G)x{t)dt. 

Suppose now that a — > +oo and /3 — )■ +oo in ([9]). Note that g{X)hua{X) — > g{X)hu{X) in the 
L^-norm. Hence 

u{Xyh^ g{-X)^ Gg{X)hu{X)dX+ / Ex(t)TGx(t)(it 

-OO ^ — oo 

We observe that, by Parseval's formula, 

cDK)] = (x(-), Gx(.)k + K-), rn(.))f; 

= {x{-),Gx{-))x + {u{-),Tu{.))^ 
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Here Gx{-) and Tu{-) denote functions with values Gx{t) and Tu{t). If the matrix satisfies 
0, i.e. G = Q- r(G), then for n(A) = g{-\)^ Q{X)b + T we have 



We now use Lemma 1 in Yakubovich (1975), Part I, §2 which asserts the following. Let U = {u} 
be an arbitrary real Hilbert space with inner product (ui, U2) and norm \ .\, with arbitrary quadratic 
functional 



defined on it, where R is a self-adjoint bounded operator in U, r £ U and p € R^. Then: a) 
if there exists an optimal point G U , i.e., a point such that ^{u^) ^ <I>(u)(Vn G U); and b) 
if there exists a number (5 > such that (u,Ru) > 5\u\'^(yu G U), then there exists an optimal 
point G U , and it is unique. 

Take U to be the space U = L^[(0, +00), R™]. We consider also the real Hilbert space 
X = L^[(0, +00), y]. Obviously, for u{t) G U, the solution of the system ([T]), ([2]) has the form 
= Qu{ ) + la where Q and / are bounded linear mappings carrying the Hilbert spaces U and 
L2(il, J"o,P,R") into X. It follows from ^ that ||Q|| < C2. It is not hard to show that, the 
functional ([3]) has the form (llOh . where ||i?|| < +00 and \r\u < +cxd. As shown above, for a = 0. 



Assume that there exists Aq G R"*^ and v G such that v*II{\q)v < 0. Let u{t) G [/ be such 
that the condition 



holds for the solution of the deterministic system -^(t) = Ay{t) + bu{t), y(0) = 0, and for the 
Hermitian form ([5]). (We find such a sequence u{t) by repeating the arguments in the proof of 
Lemma 4 in Yakubovich (1975), Part I, §2). Then {u{-), Ru{-))u < 0. If n*n(A)n > ^lup, then 
obviously (u^-), Ru{-))i/ > 6\u\fj. Thus, Theorem [T] follows from Lemma 1 in Yakubovich (1975), 
Part I, §2. 

EXAMPLE. Let n = m = 1, and write a = —A. A necessary and sufficient condition for the 
exponential stability of ([I]), ([2]) in the mean square for u(t) = is the condition 2a > > 
(see Levit abd Yakubovich (1976)). Now g[X) = {iX + a)~^, and dH) takes the form 




$(n) = (n, Ru) + 2(r, u) + p 



(10) 





. Hence, for 7 = (1 — C'^ /2a) we have that = 7G and 7 > 1. Thus, 

F(5(A)6n,n) = (-|^ + r) u\ 

For G > conditions ([6]) and ([7]) mean that F > and F > 0, respectively. For G < conditions 
([6]) and ([7]) mean that F > —'^Gl? jo? and F > —^GlP'/a^ respectively. We remark that in this 
case a stronger restriction is imposed on F than for G = 0, when ([T]) is an ordinary difference 
equation. 

2. Observe now that in (llOp 

/•oo 

{r,u) = {Qu{-),Gla)^= / Exu{t)^ Gxa{t)dt, 

Jo 

where Xu{t) and Xa{t) are solutions to the system ([1]), ([2]) when a = and = 0, respectively. 
By arguing as in the proof in Theorem [1] it is not hard to get that 

/oo 
Exu{\TGxu{\)dX 
-00 

B[g{X)bux]*Gg{X)adX+ / Bxu{t)'^T{G)xa{t)dt 
00 Jo 

(it is assumed that Xu{t) and Xa(t) are extended by zero to (—00,0) and are elements of X, and 
that Xu{X) and Xa{X) are their Fourier transforms). If G = — T(0), then 

/oo 
[5(A)6u(A)]*e5(A)EadA. 
-00 

For u G U we consider the optimization problem (u = u{-)) 

y{t) = Ay{t) + bu{t), yo = Ea. (11) 

r+00 

^iK-)]=/ [y{tyey{t) + u{tyTu{t)]dt = inm. (12) 
Jo 

Obviously $i(u) = {u,Riu) + 2(u, ri) + pi exists, with R = Ri and r = ri, for some pi G R. 
It is known (Yakubovich (1975), Part I, §1) that optimal controls in the problems ([I])-® and 
(|lip - ()12p are determined from the respective equations Ru^ + r = and Riu^ + ri = 0. Note 
that u^{t) is uniquely determined from these equations when d?]) holds, and u^{t) = h^y^{t), 
where /i is a certain nxm matrix and y'^(t) = e^'^'^^^^^^ yo. Thus we obtain the following results. 

THEOREM 2 An optimal control u'^{t) exists in the stochastic optimization problem 
if and only if an optimal control exists in the optimization problem (li j|)-([T^ ] . // condition (0) 
holds, then optimal controls in the optimization problem m)-(^and {HP, ^xist and are 

identical (and unique within equivalence). 
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REMARK 1 It can be shown that 

poo poo 

p= ^Xa{tfGxa{t)dt= / Ba^ g{-\YQg{\)d\. 

Jo J-oo 

Hence if a us a deterministic vector then p = pi. 

REMARK 2 The proofs in TheoremsUl andl^ do not change for the following cases: 

(a) Cx{t)dw{t) is replaced in by ^'j=iCjx{t)dw^^\t) , where the Cj are constant n x n 
matrices, dw{t) = [dw^^\t), ....,dw^'^^ (t)] is a standard d- dimensional Wiener process, and 
0) is replaced by the equation 

e = G + — Y, Cjg{-X)eg{X)C,dX. 

j=l -^-oo 

(b) In the definition of the spaces U and tj we replace YV^ by L^(r2, J^(0), P, R™) and C™ by 
-L^(r2, J^(0), P, C"), in ill]) we replace the condition yo = Ea by yo = a, and in front of 
the right-hand sides of and / fT2|) we place the expectation sign, E. Then p = pi and 

$i[u(-)] = $K-)]. 

REMARK 3 Equation means that AD + DA^ = -0 and = G + C'^ DC for some nxn 
matrix D. 

In conclusion the author thanks Prof. V.A.Yakubovich for valuable remarks made during the 
discussion of this work. 

References 

M.V.Levit and V.A.Yakubovich. An algebraic criterion for stochastic stability of linear 
systems with a parametric perturbation of the white noise type. Prikl. Mat.Mekh 36 (1972), 
142-148. English transl. in J.Appl.Math.Mech. 36 (1972). 

V.A.Yakubovich. The frequency theorem for the case where the state space and the control 
space are Hilbert spaces, and its applications in some problems of synthesis of an optimal control. 
I,II. Sibirsk. Mat. Zh. 15 (1974), 639-668, 16 (1975), 1081-1102. English transl. in Siberian 
Math.J. 15 (1974), 16 (1975). 



7 



